Video Mask Transfiner for High-Quality Video Instance Segmentation

While Video Instance Segmentation (VIS) has seen rapid progress, current approaches struggle to predict high-quality masks with accurate boundary details. Moreover, the predicted segmentations often fluctuate over time, suggesting that temporal consistency cues are neglected or not fully utilized. In this paper, we set out tackle these issues, aim of achieving highly detailed and more temporally stable mask predictions for VIS. We first propose Mask Transfiner (VMT) method, capable leveraging fine-grained high-resolution features thanks a efficient video transformer structure. Our VMT detects groups sparse error-prone spatio-temporal regions each tracklet in segment, which then refined using both local instance-level cues. Second, identify coarse annotations popular YouTube-VIS dataset constitute major limiting factor. Based on our architecture, therefore design an automated annotation refinement approach by iterative training self-correction. To benchmark VIS, introduce HQ-YTVIS dataset, consisting manually re-annotated test automatically data. compare most recent state-of-the-art methods HQ-YTVIS, as well Youtube-VIS, OVIS BDD100K MOTS benchmarks. Experimental results clearly demonstrate efficacy effectiveness method segmenting complex dynamic objects, capturing precise